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ABSTRACT 


This  paper  extends  the  theory  of  Programming  under  Uncertainty 
to  the  case  when  the  decision  variables  are  elements  of  a  Banach  space. 
This  approach  leads  to  a  very  natural  application  of  the  computational 
techniques  of  mathematical  programming  to  stochastic  optimal  control 
problems.  It  is  shown  that  there  exists  an  equivalent  deterministic 
mathematical  program  whose  set  of  feasible  solutions  is  a  convex  set 
and  whose  objective  function  can  be  expressed  as  a  convex  function  of 
the  initial  decision  variables.  In  the  second  part,  a  duality  theory 
is  developed  for  this  class  of  problems  and  some  of  the  relations  to 
the  maximum  principle  for  stochastic  linear  control  problems  are  given. 
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1.  INTRODUCTION 

Most  optimization  models  (programming  models,  optimal  control 
models,  etc.)  assume  that  the  model's  parameters  (coefficients,  functions, 
etc.)  are  well  specified,  either  as  best  estimates,  or  by  their  expected 
values,  and  so  on.  In  reality,  however,  these  quantities  are  subject  to 
uncertain  or  random  variations  of  various  kinds  due  to  noise,  component 
failure,  unexpected  demands,  etc.  Such  discrepancies  between  reality  and 
model  can  be  reduced  by  assuming  that  all  or  some  of  the  parameters  are 
random  variables  with  known  probability  distribution  function. 

Unfortunately,  the  complexity  of  such  models,  and  of  their  solution, 
increases  rapidly  with  the  "amount"  of  uncertainty  present  in  the  problem. 
Nonetheless,  different  approaches  and  different  techniques  have  given  us 
some  grip  on  a  certain  class  of  problems,  for  which  there  exist  now 
"efficient"  solution  methods. 

A.  Programming  under  Uncertainty 

In  1955,  G.  Dantzig  formulated  the  two-stage  linear  program  under 
uncertainty  model  [2].  The  theory  was  furthered  by  G.  Dantzig  and 
A.  Madansky  [3],  A.  Madansky  (5],  R.  Wets  [8],  and  some  special  cases  were 
investigated  by  A.  Williams  [9],  [10]  and  R.  Wets  [7], 

The  standard  form  of  a  programming  under  uncertainty  problem  reads. 


Minimize  z(x) 

■  cx 

+  E^{Min 

qy) 

subject  to 

Ax 

■  b 

Tx 

+ 

My 

-  i 

on  (E,^,F) 

x  >  0,  y  1  0. 
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where  A,  T,  and  M  are  fixed  matrices,  c,  q,  b  are  constant 
vectors,  x  and  y  are  variables,  and  £  is  a  random  vector  defined 
on  the  probability  space  (E,£7,F).  The  only  random  parameter  present 
in  this  problem  is  £•  The  decision  process  described  by  this  model  is 
a  two-stage  process  in  which  one  first  selects  x,  then  observes  £ 
and  finally  selects  y  so  as  to  satisfy  the  constraints  of  the  problem. 
The  decision  process  is  thus  divided  into  two  parts,  but  only  the  first 
one  is  of  interest  since  once  x  is  selected  and  £  is  observed,  finding 
Inf  of  qy  subject  to  My  ■  Tx  -  £,  y  >_  0  is  a  deterministic 
problem.  One  procedure  to  solve  such  a  problem  is  to  exhibit  a 
deterministic  problem,  whose  set  of  optimal  solutions  is  identical  to 
the  set  of  optimal  solution  of  our  original  problem.  In  general,  such 
a  deterministic  problem  exists,  and  it  is  shown  that  it  has  the  form  of 
a  convex  program.  To  find  an  explicit  expression  for  this  equivalent 
convex  program  is  not  always  trivial,  but  it  is  possible  to  do  so  for 
an  Important  class  of  problems  [7]. 

B.  Sequential  Decision  Processes  and  Stochastic  Optimization 

It  is  not  difficult  to  see  that  the  two-stage  programming  under 
uncertainty  problem  can  be  generalized  to  a  n-stage  decision  process 
where  we  have  a  sequence  of  decisions,  observance  of  the  behavior  of 
the  system  and  new  decisions  (corrective  action).  This  idea  is  not  new 
but  literally  illustrated  by  Dynamic  Programming.  Many  stochastic 
optimization  problems  fall  naturally  in  Ihis  framework,  even  if  sometimes 
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the  concept  of  decision  stage  may  only  be  a  mathematical  fiction, 
see  [8,  II.  A]. 

C.  The  Stochastic  Optimal  Control  Problem 

Usually,  the  stochastic  optimal  control  problem  is  also  formulated 
in  the  framework  of  a  sequential  decision  process.  But  rather  than 
dealing  with  a  finite  number  of  stages,  it  is  assumed  that  the  corrective 
actions  are  taken  at  every  Instant,  i.e.  at  an  Infinite  number  of  stages. 
To  see  this,  it  suffices  to  remark  that  a  solution  (control)  for  such  a 
problem  is  not  only  expressed  as  a  function  of  time,  but  also  as  a 
function  of  the  actual  state  of  the  system  [1],  [4].  The  observed  state 
of  the  system  consists  then  of  the  space-state  determined  by  the  control 
function  affected  by  the  Interference  of  a  random  (noise)  process. 

In  order  to  obtain  an  explicit  expression  for  the  solution  of  such 
a  problem,  or  find  an  algorithmic  procedure  leading  to  the  sclution, 
different  assumptions  have  been  made,  explicitly  or  implicitly  in  the 
formulation  of  the  problem.  From  a  practical  point  of  view,  probably  one 
of  the  weakest  assumptions  one  could  make  is  to  assume  that  the  number  of 
corrective  actions  is  finite,  either  at  fixed  time  intervals  or  at  some 
time  intervals  to  be  determined  by  the  control  system  itself. 

An  n-stage  control  system  can  be  described  as  follows:  Let  x(t) 
describe  the  space  state  obtained  by  controlling  the  system  with  u^(t) 
for  0  t  <_  t^.  Let  y(t)  be  the  observed  state  of  the  process,  i.e. 


A 


y(t)  ■  x(t)  +  £(t)  where  ((t)  is  a  random  (noise)  function.  If 
U2U)  is  the  second  stage  control  for  t^  <  t  <  t2»  we  have 
u2(t)  -  <j>(t,y1(t))  or  ^(t,u1(t)  .^(t^)  and  similarly  for  t2  <  t  <_  t3> 
we  have  u3(t)  -  \p(t,y(t2))  or  iCt.u^^Ct) , C ( t^^)  ,u2(t)  ,C(t2>)  »  and  so  on. 

This  structure  is  underlying  our  approach  to  the  stochastic  optimal 
control  problem.  We  develop  the  theory  for  a  two-stago  system  but  the 
generalization  to  an  n-stage  process  presents  no  mathematical  difficulty. 
In  Section  II,  we  derive  the  deterministic  equivalent  of  the  stochastic 
problem.  Section  III  is  devoted  to  a  duality  theory  for  this  class  of 
problems  and  its  relation  to  the  maximum  principle.  A  projected  paper 
will  deal  with  the  applications  of  the  theoretical  results  obtained  here 
to  specific  control  problems. 
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2.  THE  EQUIVALENT  CONVEX  PROGRAM 
A.  The  Problem 

The  standard  form  of  the  problem  to  be  considered  in  this 
paper  is: 

(1)  Find  inf  z(u)  ■  c(u)  +  E^{Inf  q(v[£])} 

subject  to  A(u)  ■  b 

T(u,0  +  W(vU])  •  d 

where 

•  u  is  restricted  to  lie  in  some  closed  convex  subset  U 
of  a  Banach  space  U  and,  v[£]  must  belong  to  a  closed 
convex  subset  V  of  a  Banach  space  V  for  each 

•  b  and  d  are  points  in  and  respectively. 

•  £  is  a  random  vaiiable  defined  on  a  probability  space 
(••,£7, F),  note  that  v:  E  •+•  V. 

•  c  and  q  are  continuous  convex  functionals  on  V  and 
V3  respectively. 

•  A,T,W  are  continuous  linear  operators  such  that 

A  :  U  -  91 m 
T:i/*:>f 
W  :  V  ->  SRm 

•  The  operator  E^  stands  for  expectation  of  the  inf  of  q(v[£]) 


with  respect  to 
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The  process  described  by  Problem  (1)  can  be  interpreted  as 
follows:  We  first  select  a  point  in  U ,  satisfying  the  constraints 

A(u)  ■  0  and  u  c  U,  say  u;  we  then  observe  the  random  event, 
say  £,  and  we  are  finally  allowed  to  pick  a  point  of  V  such  that 

A  A 

v  e  V,  W(v)  +  T(u,£)  *  0  and  q(v)  is  minimum.  The  decision  process 
is  thus  divided  into  two  stages.  The  second-stage  decision  is  taken, 
when  no  uncertainties  are  left  in  the  problem,  i.e.  when  the  random 
variable  has  been  observed.  This  second  stage  is  not  our  immediate 
interest  here.  Our  primary  interest  is  to  find  a  "feasible"  u  which 
minimizes  our  total  cost.  Not  only  does  our  objective  function  take  into 
account  the  immediate  cost:  c(u),  but  also  a  weighted  average  of  the 
cost  of  all  the  optimal  second-stage  decision, a  given  u,  may  lead  to. 

For  the  sake  of  simplicity,  we  shall  assume  that  (H,£7,F)  is  the 

fill  ni 

probability  space  induced  in  91  .  z  is  a  subset  of  9i  ,  F  is  a 
probability  measure  generated  by  a  distribution  function  also  denoted  by 
F  and  9  is  the  completion  for  F  of  the  Borel  algebra  in  9tm.  We 
shall  assume  that  z  is  convex.  If  this  was  not  the  case,  we  then 
replace  it  by  its  convex  hull  which  we  will  also  denote  z 
and  fill  up  &  with  the  appropriate  sets  of 

measure  zero.  Without  loss  of  generality,  we  can  assume  that  z  is  of 
full  dimension.  If  not,  we  can  change  Problem  (1)  so  as  to  include  the 
deterministic  second-stage  constraints,  into  the  set  of  fixed  first-stage 
constraints.  Then,  our  new  5  has  full  dimension.  The  probability 
distribution  function  F  is  continuous,  discrete  or  a  mixture  of  both. 
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In  view  of  Che  interpretation  given  to  (1),  it  is  easy  to  see  that 
the  second-stage  decision  (control)  variable  v  is  a  function  of  the 
observed  state  of  the  system,  viz.  d  -  T(u,0*  and  in  particular  a 
function  of  the  random  variable  £.  Thus,  v  is  itself  a  random 
variable.  This  fact  is  expressed  by  our  notation  v[£].  Moreover,  we  do 
not  make  any  assumptions  on  v  as  a  function  of  £,  e.g.,  as  to  its 

measurability.  Since  by  the  nature  of  the  model  it  is  "calculated" 
only  for  the  value  of  Q  which  actually  occurs.  We  will,  however,  show 
that  E^{Inf  q(v[£])}  makes  sense. 

In  what  follows  we  show  that  there  exists  an  equivalent  problem  to 
(1),  i.e.  a  problem  with  the  same  set  of  optimal  solutions  as  (1),  that 
can  be  expressed  as  the  minimization  of  a  convex  functional  on  a  convex 
set . 


B.  The  Second  Stage  Problem 

Once  u  is  selected  and  £  is  observed,  the  second  stage  problem 

(2)  Find  inf  q(v) 

subject  to  W(v)  -  d  -  T(u,0 
v  €  V 

becomes  a  deterministic  problem. 

Let 


(3) 


V(u,0  ■  {v  |  W(v)  +  T(u,0  *  d,  v  c  V} 
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be  the  set  of  feasible  solutions  for  (2),  and  le: 

(A)  Q(u,£)  -  Inf{q(v)  |  v  c  V(u,0) 

be  the  functional  describing  the  range  of  the  infimum  of  q(v)  as 
a  function  of  u  and  £.  As  we  shall  see  later,  we  may  restrict 
ourselves  to  the  case  when  V(u,£)  is  non-empty.  The  set  V(u,£) 
is  convex  and  closed,  but  not  necessarily  compact.  Thus,  the  functional 
q(v)  may  fail  to  achieve  its  minimum  on  V(u,0*  We  shall  assume  that 
q(v)  possesses  finite  infimum  on  V(u,0»  Such  a  condition  is  not 
very  restrictive,  because  if  for  some  u,  Q(u,£)  e  -®  for  all  £  in 
5  ,  then  z(u)  ■  -°°  and  Problem  (1)  is  of  no  interest.  Moreover,  if 
for  some  u,  Q(u,£)  equals  minus  infinity  for  a  proper  subset  of  5, 
we  could  still  hope  that  this  set  would  have  measure  zero,  and  our  problem 
could  have  a  meaningful  solution.  But  it  is  not  the  case,  since  we  shall 
show  that  if  Q(u,£)  ■  -•  for  some  £  in  E ,  then  Q(u,0  =  -®  for  all 
£  in  S.  To  do  so,  we  need  the  following  results: 

(5)  PROPOSITION:  Fix  u  and  let  V(u,0  +  0  for  all  £  in  S,  then 

Q(u,£)  is  a  convex  function  in  C  on  E. 

Proof :  For  e  >  0,  we  say  that  v  determines  an  e-  inf  of  q(v)  on 

V(u.O  if  v  c  V (u.O  and  q(v)  <_  Q(u,U  +  e. 

First,  we  shall  assume  that  Q(u,0  >  for  all  £  in  5.  Let 

CQ,  ^  e  5,  then  +  (1  -  A)^  -  ^  e  E  for  Ac  [0,1],  Let  vQ 

and  v^  determine  e  -  inf  on  V(u,£q)  and  V(u,£^)»  respectively. 

By  the  convexity  of  V  and  linearity  of  the  operators  W  and  T, 
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Avq  +  (1  -  A)v1  e  V(u,£A)  for  A  e  [0*1]* 

Then 

Q(u,Ca)  1  q (Avq  +  (1  -  A)vx) 
also,  by  the  convexity  of  the  functional  q, 

q(Av0  +  (1  -  AJv^  <_  Aq(vQ)  +  (1  -  A)q(v1> 

and  since  and  determine  e  -  inf,  we  have 

Aq (vQ)  +  (1  “  A)q(vx)  <.  AQ(u,Cq)  +  (1  -  AjQCu,^)  +  e 

l.e. 

Q(u,Ca)  <  AQ(u,Cq)  +  (1  -  AJQCu,^)  +  e. 

Since,  the  above  inequality  holds  for  any  t,  arbitrarily  close  to 
zero,  we  obtain 

Q(u,£x)  <  AQ(u,£0)  +  (1  -  AWu,^). 

Let  us  now  consider  the  case  when  Q(u,£)  is  not  finite  for  all 
K  in  5.  Without  loss  of  generality,  we  can  assume  that  Q(u,£q)  * 
If  Q(u,£q)  ■  then  for  all  N  arbitrarily  large,  there  exists 

Vq  e  V(u,^q)  such  that  q(v^)  1.  “N.  But 

Q(u,Cx)  1  q (Avq  +  (1  -  A)v1) 


and  by  convexity  of  the  functional  q 
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q(Av0  +  (1  -  Ajv^  <_  Aq(vQ)  +  0-  "  ^)q(Vj) 
and  since  Q(u,£q)  ■  -»,  3Vq  such  that 

Aq (vQ)  +  (1  -  A)q(v1)  <_  -N 
for  any  N;  thus 

Q<u,£x)  <  -N 


i.e.  Q(u,^)  =  for  A  e  (0,1). 

This  implies  that  if  there  exists  some  £  in  5  such  that  Q(u,£) 
has  no  lower  bound,  then  Q(u,0  =  for  every  C  in  the  interior 

on  5  and  Q(u,0  may  be  different  from  at  most  on  the  boundaries 

of  E. 


(6)  PROPOSITION:  If  for  a  fixed  u,  V(u,£)  +  0  for  all  in  E  and 
at  least  one  of  the  three  following  assumptions  is  satisfied: 

(i)  q (v)  is  linear  and  V  is  a  convex  polyhedral  subset  of  9fn, 

(ii)  V  is  compact 

(iii)  q(v)  is  weakly  continuous  on  V  and  V  is  weakly  compact 

(iv)  E  is  open 

then  Q(u,f)  is  continuous  in  i  on  E. 

Proof :  Since  Q(u,0  is  convex,  Q(u,0  is  continuous  on  the  interior  of 
E  (this  proves  the  proposition  under  assumption  (iv)).  Thus,  the  only  case 
of  interest  is  when  £  is  on  the  boundary  of  E.  The  proposition  under 
Assumption  (i)  is  proved  in  [8].  We  limit  ourselves  to  (ii)  and  (iii). 
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Let  F,q  c  65  and  Z,^  Cq»  where  each  ^  belongs  to  the 
interior  of  5.  Under  either  (ii)  or  (iii)  there  exists  a  subsequence 
v  k  such  that  q(v*k)  ■+  q(v^)  for  some  in  V  and  such  that 

W(v°)  +  T(u,£Q)  -  d,  where  v*  is  an  e-inf  corresponding  to  Hence, 

lim  Q(u , £  )  >_  {Inf  q(v)  |  v  e  V(u,  lim  £  )}  -  Q(u,£  ).  On  the 

i  ->•  oo  1  i+oo1  u 

other  hand,  by  the  convexity  of  Q(u,0»  we  have 


Q(u,£0)  I  lim  Q(u,^k) 

k  -+  <» 


thus 

lim  Q(u,£  )  -  Q(u,4°). 
k  -*•  • 


Remark:  The  conditions  (i),  (ii),  (iii)  or  (iv)  are  sufficient 
conditions  to  ensure  the  continuity  of  Q(u,£)»  They  are  not  necessary. 
In  general,  however,  Q(u,£)  may  fail  to  be  continuous  in  £,  as  is 
shown  by  the  following  example,  where  V  is  of  finite  dimension.  Let 
V  -  9{+,  5  -  [0,1],  q  (v)  -  q(x,y)  *  -Min  ( |  /  x  y  |,1),  d  ■  0 

W(v)  ■  x  and  T(u,£)  ■  • 


It  is  easy  to  see  that  Q(u,£)  *  -1  if  Z,  t  0  and  Q(u,0)  ■  0.  Hence, 
Q(u,£)  is  not  continuous  for  Z,  «  0. 


*■***.¥&*  *«  *  *«  S***^ 
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(7)  Corollary;  Fcr  a  fixed  u,  let  V(u,0  i  0  for  all  £  in  5. 

If  Q(u,0  *  for  some  £  in  5  and  at  lease  one  of  the  conditions 

(i),  (ii),  (iii)  or  (iv)  of  Proposition  (6)  is  satisfied,  then 

Q(u,0  =  for  all  £  in  5. 

In  what  follows,  we  shall  assume  that  either  5  is  open — or  it  can 
be  redefined  so  that  it  is  open — or  that  at  least  one  of  the  conditions 
(i),  (ii),  or  (iii)  of  Proposition  (6)  holds. 

C.  The  Solution  Set 

A  fixed  u  and  an  observed  £  determines  Q(u,0  uniquely,  then 
our  only  decision  variable  is  u.  It  is  in  this  context  that  we  examine 
the  solution  set  of  Problem  (1).  Nonetheless,  the  second-stage  decisions 
affect  our  first-stage  decision,  not  only  by  the  values  assumed  by  Q(u,0, 
but  also  by  the  restriction  that  we  have  to  limit  our  set  of  admissible 
first-stage  decision  to  those  for  which  there  exist  a  feasible  second-stage 
decision. 

(8)  Definition;  u  is  a  feasible  solution  to  (l)t  if  A(u)  *  b,  u  e  U, 
and  if  the  feasibility  of  Problem  (2)  is  independent  of  the  value  assumed 
by  £  in  5.  Let  K  be  the  set  of  feasible  solutions  to  (1).  Let 

*  {u  e  U  |  A(u)  «  b}  0  U,  be  the  set  determined  by  the  fixed  constraints . 

(9)  PROPOSITION:  is  a  closed  convex  subset  of  U. 

Proof ;  By  linearity  and  continuity  of  the  operator  A  and  convexity  of 


the  closed  set  U. 
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Let  K2  ■  {u  e  U  |  {  c  E,V(u,0  +  0}  be  the  set  representing 

the  induced  constraints.  By  induced,  we  mean  that  the  set  K2  is 
determined  by  a  condition  to  be  satisfied  at  some  later  time,  viz.: 
the  second-stage  problem  must  be  feasible  for  all  £  in  E. 

Let  K  -  {u  e  tf|V(u,0  +  0}  ,  then  K ,  -  O  K2  .  By  the 
K  (cE  ^ 

linearity  of  the  operators  W  and  T  and  convexity  of  V,  is 

convex.  Thus, 

(10)  PROPOSITION:  K2  is  a  convex  subset  of  U. 

Note  that  Introducing  the  appropriate  sets  of  measure  zero,  in 

order  to  replace  the  original  E  by  its  convex  hull,  does  not  change 

the  set  K2<  Let  E  be  the  original  probability  space  and  let  E  be 

its  convex  hull.  Let  K„  *  O  K.  and  K9  as  above.  Obviously, 

U  E  l 

K2DK2  since  the  intersection  is  taken  over  a  smaller  index  set.  Thus,  it 
suffices  to  show  that  u  e  K2  implies  that  u  e  K2»  If  u  e  K2,  then 
u  e  K2£  for  all  £  in  E,  and  V(u,0  +  0  for  all  £  in  E.  If 
?  e  but  not  to  S,  then  there  exists  £^,£2  in  “  8UC^  tliat 

l  -  A£  +  (1  -  A)£2  for  some  X  e  (0,1).  By  the  linearity  of  W  and  T 

and  since  V(u,£^)  and  V(u,£2)  are  non-empty  so  is  V(u,0»  Thus, 

V(u,U  4  0  for  all  £  in  S,  i.e.  u  e  K2» 

(11)  PROPOSITION:  The  set  K  of  feasible  solution  to  Problem  (1),  is 
a  convex  subset  of  U. 


Proof : 


k  -  k:  n  k2. 
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D.  The  Objective  Function 

To  show  that  (1)  can  be  reduced  do  an  equivalent  convex  program,  it 
now  suffices  to  show  that  z(u)  — the  objective  function  of  Problem  (1)-- 
is  a  convex  function  in  u  on  K.  Remark  that  u  e  K  implies  that 
V(u,£)  is  non-empty  for  all  £  in  ~. 

(12)  PROPOSITION:  Q(u,£)  is  convex  in  u  on  K. 

Proof :  Fix  £  and  take  uQ,  u^  c  K,  then  AuQ  +  (1  -  A)u^  *  u^  e  K. 

Since  we  assumed  that  Q(u,£)  >  -°°,  there  exists  v^  and  v^  which 
determine  c-  inf  of  q(v)  on  V(uQ,£)  and  V(u^,£),  respectively. 

Also,  by  convexity  of  V  and  linearity  of  W  and  T,  AvQ  +  (1  -  A)v^ 
e  V(uA,£).  Then 

Q(ua,£)  <_  q(AvQ  +  (1  -  AjVj^)  <_  Aq (Vq)  +  (1  -  A)q(v1)  <_  AQ(uq,£)  +  (1  -  A)Q(ulf£)  +  z. 
Since  this  inequality  holds  for  all  e,  we  have 

Q(ua,£)  <  AQ(uq,£)  +  (1  -  A)Q(ur£)  for  A  e  [0,1]. 

(13)  PROPOSITION:  Let  Q(u)  -  E^{Q(u,£}.  Then  Q(u)  is  convex  in  u  on  K. 

Proof :  The  function  Q(u,£)  is  continuous,  thus  Lebesque  measurable.  But  F 
is  a  Lebesque-Stieltze  measure  and  &  contains  the  Borel  Algebra,  thus  Q(u,£)  is 
also  F  -measurable.  Since  F  determines  a  positive  measure,  Q(u)  is  the 
result  of  a  weighted  positive  linear  combination  of  convex  functions.  Thus 
Q(u)  is  convex. 

Since  c(u)  is  convex,  we  have  shown  that  there  exists  an  equivalent 
convex  program  to  Problem  (1),  viz. 
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(14) 


Find  inf  z(u)  ■  c(u)  +  Q(u) 
subject  to  u  c  K 


where  no  random  elements  are  present  any  longer.  Nonetheless ,  two  main 
difficulties  remain  to  be  solved  before  one  could  use  efficiently  the 
techniques  available  for  convex  programs,  namely:  Depending  on  the 
structure  of  the  different  operators  of  the  original  problem,  to  find  an 
explicit  expression  for  Q(u)  and  the  set  K,  may  be  a  major  undertaking. 
As  we  shall  show  in  a  forthcoming  paper,  a  certain  and  interesting  class 
of  problems  allow  us  to  express  Q(u)  and  K  explicitly,  with  relatively 


little  effort. 
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3.  DUALITY 

A.  The  Dual  Problem 

Solution  methods  for  any  particular  problem  of  the  form  (1)  depend 
strongly  on  the  form  of  the  operators  involved.  However,  as  was  the  case 
in  linear  programming  under  uncertainty  [8],  there  is  a  dua  ’■y  theory 
which  plays  a  crucial  role. 

The  second-stage  problem,  (once  u  is  selected  and  £  is  observed) , 

Find  inf  q(v) 

subject  to  W(v)  ■  d  -  T(u,£) 

v  e  V 

and  the  equivalent  convex  program. 

Find  inf  c(u)  +  Q(u) 

subject  to  A(u)  ■  b 

u  e  k2  n  U 

are  in  the  same  form.  To  develop  the  duality  theory  for  this  class  of 
problems,  it  suffices  to  consider  the  following  simple  problem. 

(15)  Find  inf  c(u) 

subject  to  A(u)  *  b 

u  c  U  C  U% 

where  b,  c(u),  U  and  U  are  as  defined  in  the  previous  section.  We 
remember  in  particular  that  U  is  closed  and  convex,  and  A  is  a  continuous 
linear  operator  with  range  in  9im. 
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Let 

C  -  {p  -  (p0,p1,...,pm)  |  pQ  >_  c(u),  (px ....  .Pm>  "  A(u)  "  b*  u  e  U). 

and 

Q  -  { (p  ...... p  )  |  (p,,...,p  )  -  A(u)  -  b,  u  £  U} 

1  in  I  m 

(16)  Lemma:  C  is  convex. 

12  12 
Proof :  Suppose  p  ,p  e  C  and  suppose  further  that  u  ,  u  £  U  satisfy 

Pq  1  c(u1)*  (p^» • • • »P*)  *  A(u*)  -  b,  i  -  1,2. 

Let  p*  *  Ap^  +  (1  -  A)p2  and  u*  ■  +  (1  -  A)u^  for  A  e  [0,1]. 

Then 

X(pl’,,,’Pi)  +  (1  “  "  Wu1)  >  b)  +  (1  -  A)(A(u2)  -  b) 

-  A(u*)  -  b. 

Also,  by  convexity  of  the  functional  c  we  have 

c(u*)  <_  Ac(u^)  +  (1  -  A)c(u2)  <_  Ap*  +  (1  -  A)p2 
Unfortunately,  it  is  not  true,  in  general,  that  C  is  closed. 


Consider  (15)  with 


**  T  * 
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For  any  b  in  g,  inf  c(u)  *  0,  but  there  exists  no  feasible  u  such 

that  c(u)  *  0.  In  particular,  let  u*,  i*2,...  be  given  by  uj  *  6^. 

Then  A(u*)  *  0  and  c(u*)  *  — ,  i»2...  .  Thus,  we  have 

21 

p*  ■  (Pq,p|)  (—,0)  -+■  (0,0)  *  p^  where  p°  ^  C  and  each  p*  does. 


However,  we  will  need  (S  closed.  This  will  be  the  case  when  U  is  weakly 


sequentially  compact  or  if  it  is  a  convex  polyhedron  subset  of  a  finite 


Euclidian  space.  In  general,  we  see  that  we  are  essentially  seeking  to 


find  the  "lowest"  point  of  C, 

on  the  Pq  axis.  That  is,  we  can 

reformulate  (15)  as  follows: 

Find  inf 

o 

o. 

(17) 

subject  to 

p  t  LflC 

where  L  ■-  {  ( Pq.P^  •  •  •  ,Pm)  1  PA 

*  0,  i*l , . . . ,m} 

Figure  18 
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Problem  (17)  has  the  very  natural  dual. 

(19)  Find  sup  u 

subject  to  71^  =  1 

ip  -  ij  >  0  for  all  p  in  C, 

where  u  is  a  scalar,  and  tt^  is  the  first  component  of  the  m  +  1 
dimensional  vector  n. 

If  we  think  of  up  -  p  *  0  as  defining  a  hyperplane  in  9fm,  then 
there  is  a  one-to-one  correspondence  between  feasible  solutions  of  (19) 
and  non-vertical  supporting  hyperplanes  which  are  "below"  the  set  C, 
in  the  sense  that  increasing  pQ  means  up. 

Immediately,  we  have 

(20)  Proposition  (Weak  Duality):  p^  ^  p  for  all  feasible  solutions 
to  (17)  and  (19). 

Proof :  Tip  —  p  _>  0  by  (19).  Since  p  is  feasible  for  (17),  then 
~  **•  =  Pm  ■  0  and  hence  "qPq  “  p  0.  But  tt^  *  1. 

We  now  prove  the  following  intuitively  obvious  duality  theorem: 

(21)  Theorem  (Strong  Duality):  If  the  projection  (5  of  C  with  respect 
to  Pq  is  closed,  exactly  one  of  the  following  occurs: 

a)  (17)  and  (19)  both  admit  feasible  solutions  in  which  case 
inf  pQ  =  sup  p. 


b) 

(17) 

is 

feasible 

and 

(19) 

is 

not 

in 

which  case 

inf  p0  -  — , 

c) 

(19) 

is 

feasible 

and 

(17) 

is 

not 

in 

which  case 

sup  p  *  +», 

*  f  * 


\ 
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d)  neither  (17)  nor  (19)  is  feasible. 

Proof :  (a)  By  (20)  inf  and  sup  p  are  finite.  Let 

P*  ■  inf { Pq |  p  c  L  0  C}  >  -°°.  Clearly  there  exists  p*  which  belongs  to 

COL  such  that  p*  ■  p*  and  p*  is  a  boundary  point  of 

C  «  {(pQ,...,Pm)  |  p±  -  pj,  i«l,...,m,  pQ  >_  Pq,  p'  e  C}.  Hence,  there 

exists  a  supporting  hyperplane  to  C  at  p*.  Let  it  be  defined  by 

irp  -  0_>  0.  Clearly  0.  If  i?0  >  0  division  by  yields 

it p  -  p  >_0  where  it  *  *x— (it),  p  ■  for  all  p  e  C.  Since  irp*  -  p  *  0 

"0  *0 

implies  p*  *  p*  *  p;  ( tt , p )  is  optimal  for  (19)  and  inf  pQ  *  sup  p. 

If  for  every  supporting  hyperplane  of  C  at  p*,  we  have  that  it  *=  o  a 
somewhat  more  complicated  construction  is  necessary.  Let  c  >  0  be 
arbitrary,  and  let  p  *  (p*  -  c,0,...,0),  then  p  {  C.  Hence  there  exists  a 
hyperplane  separating  strictly  p  and  C,  i.e.  there  exists  it,p  such  that 
itp  -  p  <  0  and  irp  -  p  ^_0  for  all  p  in  C.  In  particular,  itp*  -  p  >  0 
but  itp  «  *  *q(Pq  ”  e)  <  "qPq  ImPHes  >  0.  Letting 

it  *■  it  and  p  *  p  *  p*  -  e ,  we  have  a  feasible  solution  to  (19)  with 
*0  0  ° 

p  *  p*  -  e.  Since  e  is  arbitrary  sup  p  *  inf  pQ  *  p*. 

(b)  If  inf  Pq  >  -<*>,  a  feasible  solution  to  (19)  exists  by  the  same 
construction  as  in  a),  but  by  hypothesis  no  feasible  solution  to  (19)  exists, 
therefore  inf  pQ  *  -®. 

(c)  Suppose  sup  p  <  4®,  then  let  p  *  sup{p|ir^  *  1,  itp  -  p  >_  0,  y  p  c  C} 

and  let  p  *  (p,0,...,0).  We  now  establish  that  p  |  C.  In  fact,  p  {  C,  for 

if  it  did  (p,,...,p  )  -  (0,...,0)  would  belong  to  (T  =G.  But  then 
i  m 
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(c (0) ,0, . . . ,0)  is  a  feasible  point  for  (17)  which  is  assumed  infeasible. 
Hence,  p  {  C.  Therefore,  there  is  a  hyperplane  separating  strictly  p 
and  C  determined  by,  say  Ti,y,  and  such  that 

ftp  <  inf  {rip  |  p  e  C}  <_  inf  { ff  p  |  p  e  C}. 

By  definition  of  C  and  since  C  is  non-empty,  we  have  the  tf^  >_  0. 

If  fig  >  0,  let  tt , u  be  ^iven  by 

it  *  tt—  •  tt  and  ■5^'  *  y  <  y  <  inf  {ftp  I  p  e  C  } 

0  0  0 

tt,u  is  a  feasible  solution  to  (19)  with  y  >  y  which  contradict  the 
definition  of  y.  Suppose  now  that  ttq  *  0.  Then  ifp  »  0,  hence 
•ftp  >  6  >  0  for  all  p  e  C.  Let  W,y  be  any  feasible  solution  to  (19), 
then  tt  »  (tt  +  Att),  y  =  (y  +  A<5)  is  also  feasible  for  any  A.  Taking  any 
A  >  0  contradicts  the  definition  of  y. 

(22)  Corollary :  If  p*  and  Tr*,y*  are  respectively  feasible  for  (17) 
and  (19)  they  are  optimal  iff 

TT*p*  ■  y*. 

Proof :  (m*p*)  -  <  (l,n*, . . .  ,tt*)  ,  (p*,0, . . .  ,0)  >  -  p*,  hence  p*  -  y* 

but  pQ  >_  y  for  all  feasible  solutions  of  (17)  and  (19)  by  Proposition  (20). 
In  particular,  p*  >_  sup  y  .  Since  y*  *  p*,  (tt* , y*)  is  optimal.  Conversely, 
inf  pQ  y*  and  since  p*  =  m*,p*  is  optimal.  On  the  other  hand,  if 
p*  and  TT*,y*  are  optimal,  i.e.  they  achieve  the  inf  and  sup  in  (17) 
and  (19)  then  by  Theorem  (21)  they  must  satisfy  tt*p*  ■  y*. 
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(23)  Corollary  (Pre-maximum  principle)  If  p*  is  optimal  for  (17), 
there  exists  a  n*  such  that: 

7T*p*  -  Min{-n*p|  p  e  C). 

Proof :  Clearly  p*  is  a  boundary  point  of  C  then  3  supporting 
hyperplane  tt*p  -  y  >_  0  for  all  p  e  C  with  n*p*  -  y  ■  0. 

TT* 

Remark:  If  in  Corollary  (23)  it*  >  0,  then  ,  p*  determines  an 
optimal  solution  to  (17);  however,  this  need  not  be  the  case  (see 
Figure  24). 


Figure  24 
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(25)  Corollary:  If  C  is  closed  and  the  infimum  in  (17)  exists  and 
is  finite,  then  the  infimum  is  attained  for  a  feasible  solution. 

(26)  Corollary:  If  L  intersects  the  relative  interior  of  C  and  (19) 
is  feasible,  the  supremum  in  (19)  is  attained. 

Proof :  [6] . 

B.  Special  Cases 

We  now  apply  the  results  of  the  last  section  to  the  original  problem  (15) 
and  examine  some  special  cases.  We  first  interpret  the  dual  problem  (19). 

For  the  special  case  (18),  it  is  equivalent  to 

(27)  Find  sup  y 

s.t.  c(u)  -  fr ( A(u)  -  b]  _  u  for  all  u  e  U. 

An  easy  lemma  is: 

(28)  Lemma:  If  c(u)  *  c*u  Is  linear  and  U  is  a  cone,  then  for  any 
feasible  *,y  for  (27)  we  have 

[c  -  A ]  (u)  0  for  all  u  e  U. 

(29)  Application  to  linear  programs:  Consider  the  linear  program 

Min  cx 
s.t.  Ax  *  b 
x  >  0. 


By  direct  application  of  (27),  its  "dual"  is 
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(30)  Find  sup  p 

s.t.  cx  -  it  [Ax  -  b]  >_  p  for  all  x  >_  0. 

Since  the  set  of  non-negative  x  is  a  cone  (28)  applies  and  we 

have 

[c  -  ttA]x  >_  0  for  all  x  ^  0.  Further,  by  taking  x  *  1^ 
i«l,...,m,  where  1^  is  the  iC^  unit  vector.  We  obtain 

c  -  ttA  ^  0  for  any  feasible  it. 

Re-arranging  (30)  we  obtain 

(c  -  ?ta)x  +  nb  b- 

Clearly  for  a  given  tt  ,  the  largest  p  is  given  by 

p  —  Trb  ■  Inf  (c  -  tt  A)  x  -  0. 
x  ^  0 

Hence,  the  dual  problem  becomes 

(31)  sup  7Tb 

s.t.  c  -  ttA  >_  0. 

To  obtain  the  usual  duality  theorem  for  linear  programming  from  (21) 
it  suffices  to  observe  that  if  sup  wb  <  ®,  then  it  is  attained  by  some 
feasible  tt,  and  similarly  for  the  primal  objective. 

(32)  Application  to  linear  control  problems:  Consider  a  dynamical  system, 

the  evolution  of  which,  is  described  by  the  ordinary  linear  differential  equations 
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■  A(t)x(t)  +  u(t)  on  the  time  interval 

[0,T]  where  x(t)  =  (x_(t),...,x  (t)),  A(t)  is  a  n  +  1  *  n  +  1 

u  n 

matrix  of  continuous  functions  and  u(t)  *  (u„(t),...,u  (t))  is  a  vector 

u  n 

of  controls.  For  simplicity,  we  assume  that  u  e  U  «{u|u(t)  e  ft 
0  <  t  <_  T,  and  u  is  measurable  and  bounded}  where  ft  is  a  closed 
convex  subset  of  9tn+^.  Further  we  assume  that 

x  (0)  *  x°  i*0, . . . ,n 

T 

and  x^(T)  *  x^  i*l,...,n. 

The  value  of  x^(T)  is  not  prescribed  and  the  problem  is  to  minimize 

xq(T)  over  all  x(t)  and  u(t)  satisfying  the  above  relations. 

T 

As  is  well  known,  x(T)  ■  Y(T)x#  +  f  Y (T)Y  ^(s)  u(s)ds  where 

J  o 

Y(T)  is  a  n  +  1  *  n  +  1  matrix  of  functions  satisfying  the  adjoint 
equation: 

Y(t)  =  -Y(t)A(t),  Y (0)  ■=  I. 

T 

It  is  easily  seen  that  ST  *  {x(T)  |  x(T)  *  Y(T)x(0)  +  f  Y(T)Y  (s)u(s)ds,  u  z  U} 

*  0 

is  convex.  Thus,  the  duality  theory  previously  developed  can  be  used. 

If  x*(t),  u*(t),  is  an  optimal  solution  to  the  control  problem,  we 

can  apply  Corollary  (23)  yielding  the  existence  of  it  »  (it  , . . . ,  tt  ) 

0  m+1 

such  that 

Ttx*(t)  *  Min{TTx  |  x  e  S^}. 

Using  the  particular  form  of  the  description  of  the  set  S^,  we  have: 
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tt  Y(t)x°  +f  YdJY'^sJu^sJds 
J0 


T 

<_  ttY(T)x°  +  j  TiY(T)Y"1(s)u(8)ds 


for  all  u  e  Q.  Thus 


0  j* uY(T)Y_1(s)  [s  (s)  -  u*(s)  ]ds . 


If  we  define  fl(t)  *  ttY(T)Y  ^(s),  it  is  easily  seen  [6]  that 

u*(s)  ■  Min  n(s)u  and  H(t)  is  a  vector  solution  of  the  adjoint 

u  e  n 

equation,  which  is  equivalent  to  the  maximum  principle  for  this  problem. 
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